Images and multi-dimensional data are usually sampled in a rectangular raster-scan pattern. Throughout the present specification, a person having ordinary skill in the art would understand that the term “sampling” may refer to downsampling or upsampling. Downsampling an image or a video frame reduces the resolution or pixel numbers of the image or video frame. On the other hand, upsampling an image or a video frame increases the resolution or pixel numbers of the image. Sampling may involve a process where all or partial samples are selected with or without any filtering. Sampling may be performed according to different sampling spacing or ratios, linear functions, or nonlinear functions. In addition, a person ordinarily skilled in the art would understand that the term “re-sampling” may refer to restoring an sampled image or video frame back to its previous state before being sampled.
Reasons for sampling image or video data are many. They include and are not limited to: (a) Easier storage since the number of samples is now smaller. (b) Smaller computational burden, since it is e.g. faster to compute a Fourier transform or perform motion estimation and compensation over e.g. a quarter of the original data compared to the original sample size. (c) Properly done sampling (e.g. preceded by low-pass filtering) may also increase the compression ratio of the data (disregarding the fact that the sample size is also smaller). (d) Sampling can help in terms of representing and categorizing the data with respect to e.g. pattern recognition or matching. Careful sampling can identify those samples that are the most critical in the representation of the image or video data. Not only is the computational burden lessened but even the success ratio of certain algorithms can benefit greatly because of sampling. (e) One may want to sample the data in such a pattern that it becomes easier to retrieve later the missing/discarded samples.